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Abstract 

Ants are known to be able to find paths of minimal length between the nest 
and food sources. The deposit of pheromones while they search for food and 
their chemotactical response to them has been proposed as a crucial element 
in the mechanism for finding minimal paths. We investigate both individual 
and collective behavior of ants in some simple networks representing basic 
mazes. The character of the graphs considered is such that it allows a fully 
rigorous mathematical treatment via analysis of some markovian processes 
in terms of which the evolution can be represented. Our analytical and com- 
putational results show that in order for the ants to follow shortest paths 
between nest and food, it is necessary to superimpose to the ants' random 
walk the chemotactic reinforcement. It is also needed a certain degree of 
persistence so that ants tend to move preferably without changing their di- 
rection much. It is also important the number of ants, since we will show 
that the speed for finding minimal paths increases very fast with it. 

Keywords: Reinforced random walks. Chemotaxis. Transport networks. 
Ant foraging efficiency. Stochastic processes. 



1. Introduction 

Transport networks play an important role in difi'erent natural and man- 
made systems. In the last years many work has been done to understand 
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collective patterns generated by the individual workers' trail laying, showing 
how complex collective structures in insect colonies may be based on self- 
organization and co-operation fTT]. Foraging ants find the shortest paths 
for initially unknown food sources in almost the minimum possible time for 
certain types of mazes ([TU] and [H]). How can an animal with only limited 
and local information achieve this in such an efficient way? Many ants, hav- 
ing only a limited individual capacity for orientation are able to select the 
shortest path between nest and food source dodging many obstacles by just 
following the pheromone trail. Just as the functioning and success of mod- 
ern cities are dependent on an efficient transportation system, the effective 
management of traffic is also essential to ant colonies. 

There are different types of ants that behave in a different way. In the 
last years, many experimental results have been developed related to, among 
others, Argentine ant [Iridomyrmex humilis) [U El |TOl [21], Pharaoh's ant 
{Monomorium pharaonis) [TH |20], Lasius niger [Hymenoptera,Formicidae) 
[21 El US] and Army ant {Eciton burchelli) [21 H li- 
lt has been proved that different ants use one pheromone (Argentine ant, 
Lasius niger and Army ant) whereas others employ three types of pheromone 
(Pharaoh's). This pheromone has a mean lifetime larger compared to the 
time spent for the ants to move from nest to food source and so ants can 
reinforce the geodesic path. 

In [TT], a series of experiments have been done with Argentine ant in 
special mazes consisting of graphs. As it is well known, Argentine ant has 
a limited individual capacity for orientation. Hence, they need to cooperate 
via pheromone trails with other ants in order to find the shortest path to the 
food source. The authors posed a model consisting on a system of ordinary 
differential equations for a graph, which is derived as a mean field theory of 
a stochastic model and it is solved numerically. 

There are not many studies concerning motion of ants in the plane. They 
are mostly concerned with the particular case of the Army ant. These colonies 
of ants are huge (may have a million of workers) and carnivores, and form traf- 
fic lanes in their main foraging columns. In [3] it is shown that the movement 
rules of individual ants can produce a collective behavior creating distinct 
traffic lanes that minimize congestion and maximize traffic fiow. This is done 
assuming pre-existing pheromone concentration with fixed profile. A general 
model of ant behavior is developed, in terms of individual-based simulation 
approach. To do so, it is studied first the behavior of individual ants in the 
absence of interactions with other ants. After so, the collective properties 
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of the model during the generation of spatial patterns are investigated. In 
this model it is shown how local interactions and individual movement rules 
can strongly influence the organization of traffic over a large spatial scale. 
Nevertheless, it does not constitute a complete model due to the fact that 
pheromone concentration is assumed to be fixed in time and the formation 
of such concentration is not explained. 

All these observations pose the mathematical problem of determining a 
minimal set of rules so that a given number of ants following them tend to 
choose shortest paths between nest and food source. From the experimental 
observations it seems that such mechanism should include the presence of 
pheromone and the persistence (tendency to follow straight paths in the 
absence of other effects). Remarkably this effect has been invoked in the 
past to explain the formation of filamentary structures in some biological 
problems such as the formation of vascular networks [19j. 

We will consider ants as random walkers where the probability to move 
in one or another direction is infiuenced by the concentration of pheromone 
near them. This kind of motion is known in the mathematical literature as 
reinforced random walks. There is a vast amount of work in this area (see 
for instance the review [ISj, the seminal paper [5] or [22] for random walks 
in graphs). The direct relation of reinforced random walks with biology was 
stressed in [T7], where general rules were found for obtaining chemotactical 
aggregation in a single point. In our study, we are mainly interested not in an 
individual random walker but rather on a large number of random walkers, 
their collective behavior, and the possibility for them to aggregate forming 
geodesic paths between two points. Our work relates to current research on 
swarming, flocking and general motions of brownian agents but with essential 
differences derived from the fact that it is chemical signals (instead of visual, 
acoustic, or other type), coupled with a directional bias in the random walk 
process, what tends to produce paths of minimal length. 

The purpose of this article is to show rigorously how the combined effect 
of reinforced random walks and persistence is able to produce the selection 
of paths of minimal length in simple networks. To do so we investigate the 
behavior of ants in a two node network and in a three node network (with 
and without directionality constraint). The paper is organized as follows: in 
Section 2 we will do some numerical experiments for a two node network and 
a three node network to understand the role of each parameter of the model. 
In Sections 3 and 4 we prove some analytical results to find the minimum 
number of ingredients that are required to obtain preference for the shortest 
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paths. Section 3 is devoted to finding the possible long-time dynamics while 
section 4 is concerned with the dynamics at early times. Finally, in Section 
5 we summarize our work and point to future directions which might be of 
interest. 



2. Numerical results 

We study numerically the collective behavior of a variable number of 
ants in networks in the form of graphs with E edges, Wi,W2, ■ ■ ■ ,We- The 
experiments are done using a Monte Carlo method with the random number 
generator binornd from MATLAB. This random number generator returns 
numbers from a binomial distribution with parameters N (number of Yes/No 
experiments) and p (probability of success). We perform a certain number 
of simulations for a given number of time steps and for a number H of ants. 

Our purpose is to explain the behavior of ants choosing the shortest path 
in terms of reinforcement and directionality constraints. We explore in detail 
what is the role of each parameter and how they affect to the collective 
behavior. 

2.1. Simulations for a two node network 

We consider a two node network with reinforcement as in figure [T] left. 
Let pwi be the probability of moving from node 1 to node 2 through the edge 
Wi and be the probability of moving from node 1 to node 2 through the 
edge W2. 

Following [6] and [10], we take the probabilities at step t to be 

n (f)- ik + ^iit)r 

P^^W + u;,{t))- + {k + U2it)y 

^ {k ^ uj^m- t{k ^ U2{t)Y ' 

where uj\{f),uj2it^ are the quantities of pheromone at each link W2 re- 
spectively at time t, k is a positive constant and a is the exponent of the 
non-linearity. The value of Ui{t) (resp. uj2{t)) is increased in one unit each 
time the ant moves along the edge Wi (resp. W2), representing the deposit 
of pheromone by the ant. 

We perform numerical experiments for different sets of parameters (fc, 
a) to create histograms with the behavior of one ant function of the 
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Figure 1: Two node network (left) and three node network (right) 

parameters. In figures |2] — f7| we consider one of tlie edges, say Wi and a 
given number n of time steps. We repeat tlie experiment a certain number 
of times and plot the number of this experiments for which the edge Wi has 
been crossed by the ants a given number of times. We can conclude from the 
graphics that 

a) If the ratio k/a is small both branches are selected equivalently; i.e. 
the ant chooses one branch at the beginning and it chooses almost all 
times this branch. 

b) If we increase the ratio k/a then the distribution becomes Gaussian. 

c) For a fixed value of k, the larger is a, the faster the histogram tends to 
a polarized state. These results agree with the rigorous mathematical 
results in and ^2] showing that for a > 1 there is a path selection 
whereas for a < 1 there is not preferentiability. 

2.2. Simulations for a three node network 

In this section we consider a three node network as in figure [T] right. We 
distinguish two cases: non-constrained and directionality constrained. In the 
constrained case, we will impose the following: if the ant is at node 1 or 2, 
it can move to the other two nodes; if the ant is at node 3 and the previous 
node is node 1, then it must move to node 2; if the previous node is node 2, 
then it must move to node 1. 
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Figure 2: Histogram for fc = 2, a = 2, rt = 100 time steps. The experiment is repeated 
1000 times and we represent the cumulative resuh. 
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Figure 3: Histogram for fc = 5, a = 2, n = 100 time steps. The experiment is repeated 
1000 times and we represent the cumulative result. 
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Figure 4: Histogram for fc = 5, a = 3, n = 100 time steps. The experiment is repeated 
1000 times and we represent the cumulative result. 
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Figure 5: Histogram for k — 5, a — 10, n — 100 time steps. The experiment is repeated 
1000 times and we represent the cumulative result. 
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Figure 6: Histogram for k = 2, a = 0.5, n = 1000 time steps. The experiment is repeated 
1000 times and we represent the cumulative resuh. 
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Figure 7: Histogram for k — 2, a — 2, n — 1500 time steps. The experiment is repeated 
1000 times and we represent the cumulative result. 
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For the case of a three node network with reinforcement (both with and 
without directionahty constraint) we have four different states for the system: 
being at node 1 (with associated probabihty pi), being at node 2 (with asso- 
ciated probabihty P2), being at node 3 coming from node 1 (with associated 
probabihty ps^i) and being at node 3 coming from node 2 (with associated 
probabihty ^34.2)- The transition probabihties are Pij probabihty of moving 
from node i to node j. If we denote by Ui{t) the quantity of pheromone at 
hnk Wi at time t {i = 1, 2, 3), then the probabihties at step t are given by: 



Pl,2(t) = 




(3) 


{k + Ui{t))^ + {k + U2it))''' 


piAt) = 




(4) 


{k + Ui{t))^ + {k + U2{t))"' 


P2,l{t) = 


{k+uj,{t)r 


(5) 


{k + tUiit))^ + ik + U3it))^' 


P2fi{t) = 


{k + UJ^{t)Y 


(6) 


{k + ui{t)Y + {k + u^{t)Y' 


P3,lW = 


{k + UJ2{t)Y 


(7) 


{k + UJ2{t)Y + {k + UJ^{t)Y' 


psAt) = 


(fc + u;3(t))" 


(8) 


{k + U2{t)Y + {k + u^{t)Y' 



In the case with directionality constraint, since an ant in node 3 is forced 
to go to node 2 if it is coming from node 1 and to node 1 if it is coming from 
node 2, one must take p^^i = 1 and ^3 2 = 1- 

We employ different program simulations to show: 

a) Temporal evolution graphics to show the relative number of times one 
ant goes through each branch without the directionality constraint. 
We conclude that reinforcement is not enough to obtain selection of 
the shortest paths, since for a sufficiently large, one particular edge is 
reinforced but it is not necessarily the shortest one (see figure [s]). If a 
is small enough, no particular branch is selected (see figure [9]). 

b) Temporal evolution graphics to show the number of times one ant goes 
through each branch with the directionality constraint. We conclude 
that directionality constraint, with a sufficiently large, is sufficient to 
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reinforce one particular path but this path is not necessarily the short- 
est one (see figure 



selected (see figure 



10). If a is small enough, no particular branch is 



11). 



We consider more than one ant. For a certain number of numerical 
experiments n, we count for each one the relative number of times that 
the shortest path is chosen with respect to the number of times that the 
longest path is chosen. The result is a set of n numbers at each time t 
that we will call ri{t),i = 1, ... ,n, so that if rj(t) > 1, then for the i-th 
experiment at time t the shortest path has been chosen most times and, 
on the contrary, if rj(t) < 1 then the longest path has been chosen most 
times. Then, at any given time t, we count the number of cases among 
the n experiments for which the shortest path has been chosen (that is, 
those experiments for which rj(t) > 1) and divide it by the number of 
times that the longest path has been chosen (that is, those experiments 
for which ri{t) < 1). The result of these calculations is a measurement 
of the preferentiability of the shortest path with respect to the longest 
path. As we can see, the shortest path is chosen more often than the 
longest path for any number of ants. Notice that the convergence to a 



constant ratio is faster when the number of ants increases. In figure 12 



we represent the limiting ratio as a function of the number of ants and 



in figure 13 its logarithm. As we can see, the ratio clearly follows an 
exponential law as a function of the number of ants, implying a strong 
reinforcement of shortest paths when the number of ants is relatively 
large. 
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Figure 8: Temporal evolution for n = 10000 time steps, a = 2 and k = 20, network 
without directionality constraint. If a is sufficiently large, any branch can be selected. 




Figure 9: Temporal evolution for n — 10000 time steps, a = 0.25 and k — 20, network 
without directionality constraint. If a is small enough, no particular branch is selected. 

As a result of the numerical simulations presented above, we conclude 
that reinforcement, persistence and a relatively large (in fact, more than 
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Figure 10: Temporal evolution for n — 1000 time steps, a ~ 2 and k — 20; network with 
directionality constraint. For a sufficiently large one of the branches can be selected. In 
particular, with only one ant it may be selected the longest path. 
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Figure 11: Temporal evolution for n — 1000 time steps, k — 20 and a — 0.25; network 
with directionality constraint. If a is small enough, no particular branch is selected. 

one) number of ants are necessary for shortest selection in our three node 
network. The effect is stronger for large values of a (stronger nonlinearities) 
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Figure 12: Ratio= ui/lj2 for the number of times that the short path is chosen with 
respect to the long path as a function of the number of ants. The parameters are k — 20, 
a = 3. Notice that the relative number of times that the shortest path has been selected 
grows exponentially with the number of ants: y = exp(0.07x). 
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Figure 13: Logarithmic Ratio= uoxjuj^ for the number of times that the short path is 
chosen with respect to the long path as a function of the number of ants. The parameters 
are k — 20, a = 3. Notice that the relative number of times that the shortest path has 
been selected grows exponentially with the number of ants: y — exp(0.07a;). 
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and increasing number of ants. In fact, the number of times that the shortest 
path is selected relative to the number of times the longest path is selected 
grows exponentially with the number of ants (see figures 12 and 13) implying 
that a large number of ants will find the shortest path quickly. 



3. Analytical results: long time dynamics 

In this section we discuss the possible dynamics at long times for the 
motion of ants in the networks represented in figure [Tj 

3.1. Network with two nodes 

We consider a two node network with reinforcement as in figure [T] left. 
Ants move one step at each time interval At that can be taken, without loss 
of generality, as At = 1. The probabilities are given by equations ([!]) and 

We recall that At is the time between two consecutive steps. We perform 
now a quasi-stationary approximation in the spirit of [T3] which consists in 
the following. Suppose that t ^ 1, then since Ui is reinforced at each time 
step, Ui is set of order t. Let us assume now that the ants perform N steps 
with NAt ^ 1 and NAt ^ t. Since the characteristic time Ui is of order 
t, we have that in the N iterations the different amounts of pheromone Ui 
can be assumed to be frozen. Therefore the evolution of the ants can be 
described with a markov chain with constant probabilities. Hence in times 
larger than NAt the occupancy times of the different nodes are proportional 
to the equilibrium probabilities for the markov chain. Since the size of the 
networks is of order one, the number of iterations needed for the system to 
approach equilibrium is of order one, and therefore the system can be assume 
to be at the equilibrium. 

We can then compute the rate of change of the Ui using these equilibrium 
distributions: 

uJi{t + NAt) - uji{t) = Npwi, 
U2{t + NAt) - U2{t) = Npw2, 



so that 



uJi{t + NAt) -uji{t) 
N 

U2it + NAt) -U2{t) 

N 



(9) 

PW2- 
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Asymptotically, using our choice of At = 1, we replace the left hand side 
of (|9| by time derivatives and then 

duji 

= PWi, 

(10) 



dt 
du}2 

~dt 



PW2 



with pwi +PW2 = 1) since we are working with probabilities, and UJ1+UJ2 = t 

Ui = 

dill 



If we do the change Ui = kQi, i = 1,2, t = kr then system (10) becomes 

(1 + ^iT 

(11) 



dr (l + fii)° + (l + n2)" 

dn2 (1 + ^^2)" 



dr 



:i+fii)° + (i+fi2)"^ 



and ^1 + ^2 = T. 

Now, in order to study the equilibrium points for system (11 ), we perform 
the change 4>i = = 1,2. Since 4>i + 4>2 = 1, we only need to take into 
account branch 0i. Hence 



dVti d d(f)i 
(r0i) = 01 + r 



(l + r0i)" 



dr dr 



and so 



dr (l + r0i)° + (l + r02)' 
1 



dr 



1+ ( 1 - 



(12) 
(13) 



If 1 <^ r, and as long as 0i is of order one, linearizing in (13) and per- 
forming the change 77 = log(r) we have 



d4>i _ 1 ^ 







(14) 



The equilibria of (14) are 



01 = 1, 01 = 0, 

Hence, the equilibrium points are 



.1 1, 



(1,0), (0,1), (-,-). 
We study in detail the behavior at each equilibrium point. 
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Case 01 = L We consider the approximation 



Introducing this value into equation ( 14 ) we have 



where we have used Taylor's expansions. 
Then, we have two different cases: 



a) If a < 1, 



<o 



dr] 



01 = ^ is STABLE, 
b) If a > 1, 



>o 



^ = (a-l)0i^0i^Ce(l"- 
drj 

01 = I is UNSTABLE. 
Case 01 = 1. We consider the approximation 

01 = 1 — 01 . 



Introducing this value into equation ( 14 ) we have 



- ^ 01 - 01 , 



drj 

where we have used Taylor's expansions 
We have the following cases: 

a) If a > 1, since 0i > 0i ,then 



7^ = -01 ^ 01 - Ce-^. 
drj 



H = 1 is STABLE. 
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b) If a < 1, since 0i < 0i , then 

— = 01 ^ (pi 
drj 

= 1 is UNSTABLE. 
Case 01 = 0. We consider the approximation 

01 = 01. 



Introducing this value into equation ( 14 ) we have 



-^01 -01. 



dr] 

where we have used Taylor's expansions 
We have the following cases: 

a) If a > 1, since 0i > 0i , then 

— = -01 (, 

ar) 

01 = is STABLE. 

b) If a < 1, since 0i < 0i , then 



ii ^ Ce-''. 



^ — 0i" ^ 01 ^ C?7 



1 

( :n 1-Q 



(ir7 

01 = is UNSTABLE. 

Therefore, a appears as a critical parameter for reinforcement of edges. If 
a < 1 then non-reinforcement will take place since the state (0i, 02) = ^) 
is stable. On the other hand, if a > 1 then one edge or the other will be 
reinforced since both (0i,02) = (1,0) and (0i,02) = (0,1) become stable. 
The result, of course, supports the numerical observations in the previous 
section. 
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3.2. Network with three nodes 

We consider a three node network as in figure [T] right with reinforce- 
ment. With the same directionality constraint as in the previous section, the 
probabilities for each state are given by equations (|3]), Q, ([s]) and 

For A; > l,t > > A^At, at a time scale [t,t + iVAt], that is under 
the hypothesis for the quasi-stationary approximation done in the case of the 
two node network, we have that: 



UJl{t + NM) - LOiit) = N{p2,lP2 + Pl,2Pl), 

uj2{t + NAt) - i02{t) = N{pi^sPi + m2), 

OOsit + NAt) - iOsit) = N{p2,3P2 + P3tl), 

where the pi,p2,P3i2 and psfi are at equilibrium. 

Asymptotically, taking At = 1 and approximating j^{oUi(t + NAt) 

UJi{t)) 



duij 



^ = 1,2,3, we get 



~dt 

dU2 

~dt 
dujs 

dt 



P2,lP2 + Pl,2Pl, 



Pl,3Pl+P3i2, 
P2,3P2+P3tU 



(15) 



with pi,3pi + p3^2 + P2,iP2 + Pi,2Pi + P2,3P2 + Psfi = 1> sluce we are working 
with probabilities, and ui + ijJ2 + uj^ = t. 

The equations for the transition probabilities of the four different states 
are then 



Pi = P2 
P2 = Pi 

P3n = Pi 

P3i2 = P2 



+ + (k + cus)' 
(A: + u;i)" 
(A; + a;i)" + {k + 102)' 

{k + U2)" 

(fc + wi)" + (fc + wa)' 

{k + U3)" 

(/c + wi)" + {k + us)' 



+ P3i2, 

+ P3n^ 



(16) 
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If we do the change Ui = kQi, i = 1,2,3, t = kr then system (15) becomes 



dr 



dr 

dn3 



{k^i + kY 
{kill + kY + {kVL^ + k)' 
{kQi + kY 



:P2 



+ 



{k^i + kY + ikn2 + k) 
{kVt2 + kY 



{k^i + kY + {kVt2 + ky 
{kQs + kY 



dr {kill + kY + (kil's + k)' 



P2 + P3n, 



and f^i + + ^3 = T- 



Similarly, for ^ 1, system (16) becomes 



Pi = P2 
P2 = Pi 
Pm = Pi 

P312 = P2 

Since r ^ 1, then by writing fli = c; 
(17) becomes 



Q1 + Q^ 

^3 



(17) 



+ P312, (18) 

+ P3tl, (19) 

(20) 

Q'l + Qf ^^^^ 
biT, i = 1, 2, 3, asymptotically the system 



+ 



Pl+P3i2, 



^P2+P3'rU 



(22) 

(23) 
(24) 
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and system (18)-(21) holds with f2j replaced by (pf. 



Pi = P2- 



P2 = Pl- 



+ P3i2, 



Psn = Pi- 



P312 P-2^a^^a- 



Substituting (27) into (26) we get 



+ 



pi- 



P2 = ^ |P2 = Pi, 



and introducing (27), (28), (29) into (23) and (24) we have 



6f + 0^ 



+ 



Pi + 



Pi + 



0^ 



Pi, 



By substituting (30) and (29) into (27) and (28) we have 



Pn-2 = Pm- 



Finally, introducing (29) and (30) into (26) we have 



Pi 1 



(1 



Psti- 



(l-202)" + 0? 

Since pi + P2 + Psfi + P342 = 1 and pi = p2,P3ti = P342 we have 



1 



Pi = P3tl- 



Plugging (|33|) into (|32|) we get 
Pil 1 



(1-202)" 



(1 - 202)" + v^2 



4,^+4,'^ 
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Now, we study the equilibrium points for system (17) as well as their 
stability. Since 01 + 02 + 03 = 1 and 02 = 03 by (30), we only take into 
account the equation for 0i. Hence 



dfli d(j)i 

^ "^1 + T—r- = 2pi 



(l + r0i)" 



dr 



dr 



'1+TC 



(35) 



Introducing the value of pi obtained in (34) into (35) and approximating 
for r ^ 1 we get 



1 



1 



dr 



l + 2( 



where we have used that 02 



1-01 



Therefore, (36) becomes 



(36) 



r- 



dr i + 2(i-")(-l + 0^i)° 
which provides an equation for 0i provided that 1 ^ r, 0i ^ r^^. 



If we do the change rj = log(r) in (37) we have 



d0i 1 




dv i + 2(i-")(^-l] 


a 01- 



To find the equilibrium points we calculate 

^ = ^ (20i)("-i) = (20i)("-^Vi + (1 - 0i)^ 
and by straightforward calculations we get 

01 = 1, 01 = 0, 01 = ^. 

Hence, the equilibrium points are 

(1,0,0), (0,2^2^' ^3'3'3^- 



(37) 



(38) 



We study in detail the behavior at each equilibrium point. 
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Case 01 = L We consider the approximation 



3 

1 



n = - + <P,. 



Introducing this value into equation (38) we have 



1 + 2 



3 



where we have used Taylor's expansions. 
Then, we have two different cases: 



a) If a < 1, 

<o 



dr] 

01 = I is STABLE, 
b) If a > 1, 



(a - 1)01 ^01 ^Ce^-I"-^!"). 



drj 

!)i = 1 is UNSTABLE 



>o 

(a - 1)01 0~i ^ Ce^l"-'!''). 



3 

Case 01 = 1. We consider the approximation 



Introducing this value into equation (38) we have 



l + 0i = -2^->i + 



dr] 

1 + 21 



1 01 

2 1-6 



where we have used Taylor's expansions. 
We have the following cases: 
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a) If a > 1, since <pi ^ <pi , then 



'1 



dr] 

01 = 1 is STABLE, 
b) If a < 1, since 0i <^ 0i , then 

OX] 

01 = 1 is UNSTABLE. 
Case 01 = 0. We consider 0i small. Introducing this value into equation 



(38) we have 

rf01 1 



, . ^-01=^1-2(1-") i-0r" - 01, 

1 + 2 \ 



where we have used Taylor's expansions. 
We have the following cases: 

a) If a > 1, since 0i 0", then 



— = -01 ^ 01 ^ Ce 
dr] 



01 = is STABLE, 
b) If a < 1, since 0i <^ 0", then 

^ = l-2(i-°)0^'^^0i^Cr/if^. 
drj 

01 = is UNSTABLE. 

As a conclusion of the analysis of the three node network, if a < 1 then the 
three edges are run with identical probability since the only stable equilibrium 

is (01,02,03) = (I'I'I)' while for a > 1 the states (0i,02,03) = (1,0,0) 
and (01,02,03) = (0, \, 1), corresponding to the shortest and longest paths 
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respectively, are the stable ones. This result implies that for a > 1 one 
particular path, the short or the long one, will be reinforced and our random 
walker will end up walking on it with a probability that tends to one as 
time goes to infinity. Nevertheless, the analysis does not provide a reason 
for the shortest one to be selected preferably with respect to the longest 
one. In the next section, we will see that such a selection takes place in 
the first stages of the evolution when reinforcement is still very weak and 
provided that more than one ant are running through the network. In order 
to perform this analysis, we will linearize the probabilities given by (|3|-([6]) 
using as a small parameter a/k and solve the resulting evolution problem. By 
assuming a/ /c ^ 1 we are considering the case where reinforcement remains 
very weak up to times when Ui = 0{k). We will show that the difference 
in the amount of pheromone between the shortest path and any of the links 
in the longest path has a probability distribution that evolves according to 
a convection equation. The convection velocity, when there is more than 
one ant, is always in the direction of increasing the value of the difference in 
the amount of pheromone and hence the shortest path will be increasingly 
reinforced. This breaking of symmetry occurs faster with increasing number 
of ants due to the fact that the convection velocity grows very quickly with 
the number of ants. 

4. Analytical results: early time dynamics with weak reinforce- 
ment 

4.I. Reinforced and non-reinforced network with one ant 

Considering the three node network in figure [l] right for one ant, we have 
two possible states: 

• The ant is at food source (node 2) or nest (node 1), case A. 

• The ant is at node 3, case B. 

To simplify the analysis we restrict the problem to times t so that Ui = 
0{t) <^ K This corresponds to the case where reinforcement is still very weak 
due to the fact that cuj <C /c in formulas (|3])-([6]). We can then approximate 
the probability pi^2 in formula ^ by 
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(A; + wi)" + (/c + wa)" 2 + a{f + f) 

1 A uji-uj2\ 1+eA 
- 1 + a ; = , 

2 V 2A; y 2 ' 

where £ = and A = Ui — CO2. 
Analogously, 

1-eA 1 + eA 1 - eA 

Pl,3 ~ ^ , P2,l ~ ^ , P2,3 ~ — ^ — • 

Notice that we have approximated U2 = ujs since both edges W2, W3 are 
run the same number of times due to the directionality constraint imposed. 
Considering the cases A and B, we can describe any possible evolution as a 
sequence of the following states: 

1. From state A to state A: p = A ^ A + 1. 

2. From state A to state A passing through state B: p = A — ^ A — 1. 
Then the master equation for the probability is: 



p(A, AT + 1) = 1(1 - eA)p{A + 1, iV) + 1(1 + eA)p{A -1,N). (39) 

Equations at 0{e^) 
From (39) we have that 

p{A, N + l) = lp(A + 1,N) + lp(A - 1, AT). (40) 



If we subtract p{A, N) at both sides in (40), we get 

|^p(A,Ar+l)-KA,iV)^l^. 

This is a diffusion equation without transport terms (i.e. terms involving 
1^) and hence the solution is such that if p(A, 0) is centered at A = Aq then 
p{A, t) will also be centered at A = Aq. Therefore, the maximum probability 
will always be at A = Aq and hence no path will be reinforced. 

Equations at 0{e^) 
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If we subtract p(A, A^) at both sides in 39, we get 

dp 1 d^p dp 

where we have done the following approximations 

^^p(A,iV + l)-p(A,iV), 

dp _ j9(A + 1, A^) - p(A - 1, A^) 
M 2 ' 

p(A + 1, AT) + p(A - 1, AT) - 2p(A, N). 



d'^p 



9A2 

If p(A, 0) is centered at A = Aq then the presence of a convective term 
with velocity eA will produce a shift of the movement of p(A, t) to increasing 
(if Aq > 0) values of A or to decreasing (if Aq < 0) values of A. Hence, 
one of the paths, the short or the long one, will be reinforced depending on 
the initial condition. This agrees with our previous numerical simulations 
concerning the fact that only one ant is able to reinforce one of the paths 
but more than one ant is necessary to actually reinforce the shortest one 
preferably. 

4-2. Reinforced and non-reinforced network with two ants 

In this section, we consider at the same time both the reinforced and 
non-reinforced cases for two ants in a three node network with directionality 
constraints. 

We show that it is enough to consider both directionality constraint and 
reinforcement to reproduce ant's behavior concerning choice of the shortest 
path. 

Considering the three node network in figure [T] right for two ants, we can 
classify any state into these four different states: 

1. Both ants at nest (node 1) or food source (node 2), case A^] 

2. One ant at nest and the other at food source, case 

3. Both ants at node 3, case B\ 

4. One ant at nest/food source and the other at node 3, case C. 
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Figure 14: Different ant's states. State A'^ also represents the situation where both ants 
are at the food source. The state C also represents the situation where one ant is at the 
food source and the other at node three. 

By employing these 4 states, we can describe any possible evolution as 

a sequence of such states. The probability of being at one of the states at 
a certain time will depend on the probabilities of having previously been at 
other states. In order to arrive to a simple way to compute such probabilities, 
we find a representation of the evolution as a markovian process where one 
can write the probability to reach a certain state at time A^" + 1 merely as a 
function of the probabilities to be at each state at time N. The directionality 
imposed in the problem reduces drastically the number of possible transitions 
from one state to the other so that any evolution of the system can be viewed 
as a sequence of "syllables" 

A+r(^j) A+ A+C^'^j+'^) A- A-C^'^j+'^) A+ A-P^'^f) A- 

where C" means that state C is repeated n times. Notice that any of the 
syllables leaves the system at or A~ state and implies a certain change 
in the relative amount of pheromone A and hence, in the probabilities for 
the transition from one state to the other. Therefore, we can compute the 
probabilities of being left in state ^+(resp. with a certain value of A 

after the syllable £ + 1 as a function of the probabilities of each syllable to 
occur and the change in A that they produce. These can be easily computed, 
by induction, to be: 

1. From state ^+ to state A^: p = ^^^4^, A h-^ A + 2; 
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2. Prom state A" to state A': {i+^^^r ^ A A + 2; 

3. Prom state A'^ to state A'^ passing through state B: p — 

A-2; 

4. From state A" to state A~ passing through state B: p — 
A ^ A - 2; 

5. From state A~ to state A'^ passing n = 2j + 1 times through state C: 

- ^ + - ^^1^ + ^(^')'^' = 0' 1' ■ ■ ■ ' 

A^ A-(2j-l),j=0,l,...; 

6. Prom state A'^ to state A^ passing n — 2j times through state C: 

1 272-67 + 2 ,27-2 

A^ A-(2i-2),i = l,2,...; 

7. From state to state A^ passing n = 2j + 1 times through state C: 

1 272 -4:1 + 1 ,27-1 

A^ A-(2j-l),j = 0,l,...; 

8. Prom state A~ to state passing n — 2j times through state C: 

1 272-67+4 ,27-2 

A^ A-(2i-2),j = l,2,...; 

If we set 

p{N + l,{A,A-))^PN+i{A-), 

piN + l,{A,A+))=pN+iiA+), 

for the probabihties of having a relative reinforcement A and end after the 
N + 1 syllable at state A' or A'^ respectively, then one can easily write 
master equations for pjv+i(A~) and pAr+i(A+) using the probabilities of each 
syllable (points 1 to 8 above) . If we perform the approximation 

p(A+'- + 5)=p(A+'-) + (5^(A+'-), 
28 



in the master equations, then after straightforward calculations we arrive at 
the following equations for pjv+i(A~) and pn+i{^'^)'- 



where 



and 



where 



2 1 62 

pjv+i(A-) = -pjv(A-) + -pjv(A+) - — £pAr(A-) 

- ^^ep^{A^) - (5A)^-^^-^(A-) (41) 
-(5A)^.^^-^(A+) + 0(A^s^), 



1 32 64 ^ 

= \ e H eA, 

9 27 27 

(^^)---- = 9 - r + 27^ + E 4^ 

1 34 17 ^ 
= 9 + 27^+27^^' 



1 2 19 

p^^+l(A+) = -piv(A-) + -piv(A+) - — £P7v(A-) 

- ^epAr(A+) - (<5A)^-^^+^(A-) (42) 
-(5A)^+_^+^(A+) + 0(A^£2), 



1 2 37 , , ^ (2i-l)' 

1 31 17 , 
= 9 + 27^+27^^' 

. 1 2 44 ,^(27-2)^ 

1 38 64 ^ 

= \ e H eA. 

9 27 27 
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Next we proceed to solve equations (41), (42) both for e 
Equations at 0{e^) 



and e > 0. 



From (41) and (42) we have that 



PAr+l(A' 



9 ^ 



1 dp 



N 



9 



(43) 



PAr+l(A" 



^pjv(A-) + ^p^(A^ 



1 dp 



'N 



9 9A 



(A- 



9 9A ^ 



By adding (43) and (44) we get 



PjV+I =PAr+l(A ) +J9Ar+i(A+) = pAr(A )+Piv(A" 



Subtracting (44) and (43) we have 



6p 



N+l 



PAr+l(A+) -pAr+i(A 

2dpN 



1 



m(A- 



1 



9 5A 



(A- 



9 9A ^ 



3P^(A^ 
3^^^+ 9^- 



Since 



5pAr+i - 5p 



N 



dSp 
~dt' 



then (46) becomes 



d5p 
~dt 



2 , 2d5p 
-3^^+9aA' 



and so 



5p = exp( 



|t)5po(A + ^t), 



(44) 



(45) 



(46) 



(47) 



(48) 



where Spo is the initial data. This solution shows us an exponential decay for 
the difference of probabilities pj^f^A'^) —pj^{A') while the sum of probabilities 



remains constant for any A (45). As a consequence, no preferential selection 



of any edge takes place. Directional persistence is not sufficient at this order 
for shortest path's selection. Next, we will discuss whether lower order terms 
are able to explain this preferential selection or one needs to invoke different 
effects. 

Equations at 0{e^) 
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Subtracting pAr(A ) from (41) and p^^A'^) from (42) we have 



dp 1 1 



-/32 64 . 
P )a-Pa(^£+;^£A) 



27 



, .34 17 62 
^^(27"+27"^)-27"^" 



10 
27 



ep'^ 



(49) 



~dt 



where 



dp 
~dt 



^{p -p^)-^{p -p+)a -PA(pe+ ^^A) 



, .38 64 . 



p+ =p^(A+), 
P7V+l(A") -p7v(A" 



19 
27 



ep 



71 
27 



ep' 



p 
dp'^ 



9A 



(A^ 



dt 



Pn{A-), 

p^+i(A+) -pjv(A^ 
dpN 



dA 



(50) 



Adding equations (49) and (50), putting p = p^ + p and making the 
approximations p+ — n- — 



2 — ^ we have 



dp 
di 



-^{5e + 6eA)p/^ - ^e{Sp)A - 3£p 
((-^e-3eA)p)^-i5(5p)A, 



where we have also approximated the derivatives by 



Pa 



^PA + ^{Sp)A, 



Pa = \pA - ^(^p)a, 



(51) 



and 5p is given by (48). Neglecting 5p which is exponentially decreasing in 



time (see equation (48)), equation (51) becomes a transport equation that 



can be solved using the characteristics method. The characteristics are the 
solution of 

dA 5 5 

:€ + 3£A ^ A = -(Kexp(3£:t) - 1), 



dt 



6 
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and hence 



p = exp(-3£t)po(A - -(exp(3£:t) - 1)) 

5 5 
~ exp(-3£i)po(A - -Set) = exp{-3et)po{A - -et), 
o / 



(52) 



since -C 1 and where po is the initial data. 

Notice that the probabihty distribution shifts towards increasing values 



of A at a velocity ^e, (52). Hence, we conclude that shortest path (the one 



producing increase of A) is progressively reinforced. This result provides an 
analytical proof support for the fact that both reinforcement and persistence 
are sufficient to produce shortest path selection at least for two ants. Remind 
that such shortest path selection was not possible with only one ant. In the 
next section we will consider the problem for larger number of ants. 



4-3. Non-reinforced network with H ants 

Now, we consider the three node network in figure [T] right but for H ants. 
Our analysis with two ants without reinforcement lead, from formulas (41) 



and (42) to a system that can be written in the form 



P{A' 
P(A^ 



A 



N+l 



P(A- 
P(A^ 



B- 



d ( P{A' 



N 



OA V ^(A^ 



N 



where 



A 



3 3 
1 2 



B 



3 3/ \ 9 9 

and the states A~ and A'^ correspond to 1 or 2 ants in the nest respectively. 
Notice that the elements of both the rows and columns of matrix A sum 
one and they are positive (since they correspond to probabilities). Moreover, 
matrix A is symmetric. These properties are also verified when considering 
H ants and, therefore, H possible states A^,A'^,...,A^ corresponding to 
1,2, ... ,H ants at the nest respectively. We can also write the system 



/ ^(A') 
P(A2) 



A 



( ^(A') 
P(A2) 



V ^(A^) / 



B 



d_ 
dA 



( ^(A') 
P(A2) 



N+l 



\ 



(53) 



N 



\ ^(A^) / 



N 
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with A a symmetric matrix with positive entries such that any row and 
column sums one. This characterizes A as an stochastic matrix which is, 
moreover, symmetric. The Perron-Frobenius theorem imphes then that there 
exists an eigenvalue A = 1 and all other eigenvalues A are such that |A| < 1. 
Since the matrix is symmetric, such eigenvalues are real. The eigenvec- 
tor corresponding to the eigenvalue 1, called the Perron-Frobenius eigen- 
vector, is (1,1,...,!)^. This implies that A = Q~^DQ with D the ma- 
trix of eigenvalues. The lack of reinforcement implies that the probabilities 
P(A^), P(A^), . . . , P{A^) are indeed independent of A and therefore we will 
denote them as -Pi, -P2, 
the form 



Ph- For all these reasons, (53) can be written in 

(54) 



Af+l 



DP^ 



N, 



where Pn+i = Q {Pi, P2, ■ ■ ■ , Ph)n+v 



Notice that (54) is the discretized version of the following system of equa- 
tions _ 

dP 

^ = - np, (55) 

with solution P{t) = exp{{D — I)t)PQ. Notice also that all components 
of D — I, except for the first one (corresponding to the Perron-Frobenius 
eigenvalue A = 1 for A) are negative and hence 



(\ ••• \ 

e(^2-i)i : 
; ■•. 

\ ■ ■ ■ e(^«-i)* j 

with Xj < 1, j = 2, . . . , H. The eigenvectors ej corresponding to eigenvalues 
Xj , j = 2, H, are orthogonal to the Perron-Frobenius eigenvector. Hence 



the vectors with all entries equal to except for a 1 at position k and a 
— 1 at position /, which are orthogonal to the vector (1, 1, 1)-^, are linear 
combinations of the eigenvectors ej, j > 1. This implies that 



Pk,N — P 



LN\ 



< Ce 



-inf,>i{|{A,-l)|}t 



for all k, I. 



(56) 



Since the probability distribution converge to equilibrium exponentially fast, 
we will not make distinction among different states in what follows and we 
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will merely write equations for 



H 

or, in other words, for the component of the probability vector (Pi, P2, • • • , 
on the Perron-Frobenius eigenstate (all the other components converge ex- 



ponentially fast to zero by (56)). 



4.4- Reinforced network with H ants 

Finally, we consider the three node network in figure [T] right with rein- 
forcement and for H ants. As in the case for two ants, we can decompose 
the evolution as a sequence of syllables starting and ending in a state A at 
which all ants are at the nest or the food source. All the possible syllables 
are of the form AA^ ABA and AC^j)A, where the state B consists of all ants 
in the node that is not nest nor food source and state C can be any possible 
combination of ni ants at the nest or food source and n2 ants at the other 
vertex. Since there are arbitrary long sequences of C states we will denote by 
n[l),n[2), . . . the number of ants that are at the nest or the food source at 
each of the C states of the sequence. Similarly, we denote by n2l),n22), . . . 
the number of ants that are not at the nest nor at the food source. Notice 
then that n[j) + naj) = H. Following the same steps as in the case for two 
ants, we can then write the following equation for the probability: 

p{A,N + l) 



1 /^V H-nt'^ \\ (57) 




(1 + ^2f5,4i),.,.,„(s)^(A)) p{A - {{S + l)H - 3x), N), 

where the sum goes over all S from 1 to 00 and over all the S"— uplas 

such that 1 < n^^^ < H, nf^ + n}^^'^ < H.'ik > 1, 
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1) „(S) 



[n 



(1) 



+E 



(A) 

4^')(A- (5 + l)i/ + 3x) 
i 



(i) 



2 J 



k=l 



712 +122 



+ {A-{S + l)H + 3x] 



Applying Taylor's expansion in (57) and using the relations 



1 



2^ ' /-^ 2(^+^)^-^ 
{S+l)H-3x 



n 



2{S+l)H-x 



n 



H-n 

U) 



0, 



(58) 



(59) 



(60) 



(see remark at the end of this chapter for a proof of this formulas) we arrive, 
keeping up to 0{e) terms, at the equation 



p(A, N+l)= p(A, N) + c(A)£^(A, iV), 



(61) 



where 



^(A) = -E 



{S + l)H -?,x 

2{S+l)H-x 




H 



n 



(i-i) 



(4^^ + 



{5,n^^^...,4^'} 



(A). 



(62) 



Equation (61) is a discretized version of the transport equation 

9p(A,t) 



dt 



The constant c(0) is computed numerically from formulas (57) and (58). 
Since we are considering the early times when the maximum of the proba- 
bility distribution is close to A = 0, it is the convection produced by c(0) 
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what breaks the symmetry and shifts the probabihty distribution towards 
increasing/decreasing values of A at a velocity c(0) provided it is strictly 
positive/negative. Our numerical computations yield the values for the ve- 
locity c(0) = 11.2476 for 3 ants and c(0) = 28.2320 for 4 ants. Hence, the 
shortest path will be progressively reinforced and this will occur at a velocity 
that increases with the number of ants. 



4.4- 1- The limit e = 



By considering e = in (57) we return to the problem without reinforce- 



ment and the equation for the probability is: 
p(A. N+l) = ifp(A -H,N) + -Lp(A + H, N) 



+ E 



2{S+l)H-x 



n 



H 



H - (nX^' + n 



n. 
(i) 



p{A-{{S + l)H -'ix),N) 

(63) 



where the sum goes over all S from 1 to oo and over all the S"— uplas 

< Hyk> 1 



n 



(1) „(2) 
2 ; '*2 5 



and X = 



[k) 



such that 1 < rS^'' < H and n)^' + n 



(k) 



(k+i) 



k=l 



n. 



. We suppose that 



(0) 



n. 



{s+i) 



0. 



Applying Taylor's expansion in (63) we have that 



p(A,Ar + l) 



+E 



1 



^p(A,iV) + i^p(A,iV) 



E 



2{S+l)H-x 

{S + l)H-3x 



2(5+l)H 




p{A,N) 



(i-i) 



dp 
dA 



(64) 



As a final remark, we note that without reinforcement the convection 



velocity vanishes by formula (60), p(A, N + 1) = p{A, N) by formula (59) 



and hence no path selection takes place. Formula ip9^ follows from the fact 



that the sum of all the probabilities must to be one and formula (60) must 
necessarily be true due to the following: 
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1. If there is not reinforcement, the movement of just one ant does not 
affect the others. 

2. If the initial position for all ants is at node 1, we have with probability 
1 that all the ants will be at node 1 at a subsequent time t. 

3. The mean A change produced by an ant with initial position at node 
1 and coming for the first time also to node 1 is 

(5A) = 2pi_2,l + (-2)^1,3,2,3,1 + 0pi,3,2,l + 0pi,2,3,l 

= 2^ + (-2)^ + + = 0. 

When all the ants meet at node 1, each ant has done a certain number of 
elementary paths: (1, 2, 1), (1, 3, 2, 3, 1), (1, 3, 2, 1), (1, 2, 3, 1). Since the mean 



change {SA) = 0, then (60) must be true. 



The computational time grows exponentially with H so that we can only 



compute a few values of summands in (62) (for A = 0) when H < 5. Never- 
theless, the tendency to reach larger values of c(0) can be clearly appreciated 
at least for if = 5, 6, 7. 



5. Conclusion 

We have presented a model for ants to simulate their behavior when for- 
aging. It is well know that social insects, as for example ants, leave a trail 
to coordinate the group and to communicate to each other. This pheromone 
plays an important role to recruit the individuals and reinforce the shortest 
path between nest and food source. We have shown by means of numerical 
simulations and analytical arguments that in order for the ants to follow the 
geodesic path in a two or three node network, it is necessary not only to in- 
voke the pheromone-induced reinforcement but also to have a directionality 
constraint. Such constraint is so that ants prefer to maintain their direc- 
tion of motion to turn back and return. Furthermore, more than one ant is 
also needed to reinforce the geodesic path, with the velocity of reinforcement 
increasing exponentially fast with the number of ants. We expect that the 
combined effect of reinforcement and persistence is able to induce the forma- 
tion of ant trails of minimal length not only in simple networks, but also in 
more complex networks, in the plane or in surfaces of general topology. This 
is the object of our current research and results will be presented in future 
publications. 
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